Experiments are typically conducted to determine empirically if there are relationships between two or more variables. An experiment may begin with the formation of one or more hypotheses positing that there is a relationship between one or more independent variables and one or more dependent variables. For example, a researcher at a pharmaceutical company might formulate a hypothesis that the amount of a new drug that patients take will be related to the blood pressure of patients. Independent variables are the variables defined or manipulated by the experimenter during an experiment (e.g., the amount and/or frequency of a drug administered to patients). Dependent variables are the variables posited to be predicted by the value of the independent variable (e.g., the blood pressure of patients). The experimenter then conducts an experiment to determine if there is indeed a relationship between the independent and dependent variables (e.g., if the amount of a drug patients receive is related to the blood pressure of patients in a pharmaceutical experiment).
Confounding variables (things that could vary systematically with the levels of the independent variable) may also influence the dependent variable. These confounding variables are not of primary interest in the experiment, yet can influence the dependent variables and therefore obscure an accurate cause and effect relationship between the independent and dependant variables. The experimenter is trying to understand the causal relationships between the independent and dependent variables, however, these confounding variables can render the results of an experiment uninterpretable. Some examples of confounding variables include Hawthorne effects, order effects/carry over effects, demand characteristics, and/or any other factor that could vary systematically with the levels of the independent variables, e.g., such as the body mass of a test subjects in the pharmaceutical experiment discussed above. Confounding variables make it difficult or impossible to know which factor (variable) caused any observed change in the dependent variable(s). And thus, the existence of confounding variables that are not properly controlled during the experiment renders it difficult or impossible to make statistical inferences about causal relationships between the independent and dependent variables. Various types of experiments may be distinguished by the manner and degree to which they are able to reduce or eliminate the effects of confounding variables. The term “true experiment” denotes an experiment in which:
1. There are at least two levels of an independent variable.
2. Samples are randomly assigned to levels of the independent variable.
3. There is some method of controlling for or eliminating confounds.
Experiments that lack any of the above three characteristics are not true experiments, and are often referred to as quasi-experiments or correlational studies. Although, the term experiment is used to describe studies that lack any of the 3 characteristics above, those skilled in the art of experimental design will recognize that these studies are actually quasi-experiments or correlational studies. Only true experiments allow statistical inferences to be drawn regarding the causal relationships between independent and dependent variables. Quasi-experiments and correlational designs may allow relationships between independent and dependent variables to be established, but it is not possible to determine whether those relationships are causal. Various types of experimental designs (including true experiments) have been described, for example, in Campbell, D. T., & Stanley, J. C. (1963) Experimental and quasi-experimental designs for research, Chicago: Rand McNally. Only true experiments deliver results unaffected by confounding variables and can empirically determine the direction and strength of causal relationships. However, the complexity of designing a true experiment that appropriately controls for or eliminates confounding variables may be significant. Manually conducted true experiments require time, resources, statistical expertise and deep knowledge of the scientific method, which often prevent wide use today.
It is desirable to design experiments that have a sufficient degree of internal and external validity. Internal validity refers to the confidence with which one can conclude that any change in the dependent variable was produced solely by the independent variable and not due to any extraneous or uncontrolled variables. For example, a blood-pressure drug experiment in which the control group took a placebo pill would be more internally valid than an experiment in which the control group was not given a placebo (because without giving the placebo, the level of the dependent variable (blood pressure) could have been produced by the act of taking a pill or could be caused by the actual chemical composition of the drug) External validity refers to the extent to which the results of an experiment are generalizable or transferable. For example, a blood-pressure drug experiment in which the results can be generalized to all people would be more externally valid than an experiment in which the results could only be generalized to those who have already had a previous heart attack. Designing a true experiment having sufficient internal and external validity may be daunting for investigators who have only a limited knowledge of the statistical and experimental design principles. The expert system described herein provides investigators with a tool for designing experiments without requiring extensive knowledge of the underlying theory of true experimental design. The expert system also aids investigators in conducting the experiments, collecting data, statistically analyzing data, and interpreting the results of the experiments.